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Technology  Assessment:  Man-Machine  Interfaces 


I.  Historical  Review 

i 

■^Interfaces,  also  called  man-machine  interfaces  or  input-output  (l/0)mechanisms, 
are  the  means  by  which  humans  communicate  with  computer  resources  and  applications. 
In  the  first  two  decades  of  computer  development  (1950’s  and  1960’s),  interfaces 
designed  for  general  purpose,  end-user  application  were  very  primitive.  Card  readers, 
card  punch  machines  and  hard  copy  printer  terminals  were  the  predominant 
man-computer  interface  devices.  Since  the  late  1960’s,  keyboards  and  monitors  have 
been  the  dominant  communications  interface  between  the  computer  and  the  user  in  an 
increasingly  more  interactive  environment.  Throughout  most  of  this  period  of 
development,  the  user  has  been  required  to  have  significant  training  and  knowledge  of  the 
specific  hardware,  operating  system  and  application  in  order  to  use  the  computer.  As  the 
use  of  computers  began  to  proliferate  during  the  1970’s,  the  demand  for  simpler,  more 
easily  learned  interfaces  began  to  grow.  (_J7^  )^~ 

This  demand  has  effected  significant  changes  in  the  design  of  interfaces, 
particularly  through  the  influence  of  ergonomic  considerations  in  these  designs.  The 
current  trend  in  man-machine  interfaces,  addresses  sensory  capabilities  beyond  the 
tactile,  particularly  the  use  of  speech  and  vision  as  a  medium.  This  paper  will  address 
those  trends  and  examine  the  near  term  implications  of  these  trends  on  the  automated 
systems  developed  in  the  near  term  (3-5  years),  short  term  (6-10  years)  and  long  term 
(2010). 

Many  interface  mechanisms,  both  hardware  and  software,  that  are  available  today 
were  actually  developed  and  have  been  widely  used  in  the  academic  and  research 
communities  for  many  years.  Bit  mapped  graphics — a  high  speed  method  for  screen 
display  in  which  the  enire  screen  image  is  stored  explicitly  in  an  area  of  memory  called 
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a  bit  map,  where  each  screen  pixel  (dot  of  light)  location  has  a  one-to-one  mapping  to  a 
counterpart  in  memory  — light  pens  and  mice,  icon  graphics  and  windowing  interfaces 
have  been  used  in  these  communities  for  15  years  or  more.  During  this  period,  interface 
mechanisms  were  hand-crafted  on  an  ad-hoc  basis  for  the  specific  application,  usually 
on  powerful  workstations  or  terminals  to  powerful  mini-  or  mainframe  computers. 
These  interfaces  were  designed  generally  to  increase  productivity  by  minimizing  the 
level  of  “housekeeping”  chores  necessary  to  use  the  system.  Over  a  period  of  time, 
many  of  these  techniques  were  collected  into  integrated  “toolkits”  that  became  more 
commonplace  within  the  academic  and  research  communities.  Most  of  these  toolkits 
included  the  use  of  a  mouse,  icon  graphics,  and  sophisticated  methods,  such  as 
hypertext,  to  access  different  levels  of  information. 

Hypertext,  or  its  more  generalized  form,  hypermedia,  is  an  approach  to  information 
management  in  which  data  is  stored  in  a  network  of  nodes  connected  by  links.  These 
nodes  can  contain  graphics,  text,  video  or  other  forms  of  data,  and  are  generally  viewed 
and  manipulated  through  a  windows-like  interactive  browser.  It  has  only  been  over  the 
past  decade,  that  a  trend  toward  “user-friendly”,  more  intuitive  interfaces  like  these  that 
provide  both  easier  accessability  to  the  application  and  greater  transparency  to  the 
hardware  and  operating  system  for  the  user  has  become  prevalent.  Much  of  the  impetus 
for  this  has  been  the  widespread  proliferation  of  end-user  computing  that  was  effected 
by  the  “micro-computer  revolution"  of  the  early  1980’s.  This  proliferation  of  relatively 
inexpensive  microcomputers  has  exposed  a  growing  number  of  technically 
unsophisticated  users  to  increasingly  more  powerful  computer  resources.  This 
environment,  in  which  a  greater  number  of  relative  computer  amateurs  are  being 
required  to  apply  increasingly  greater  emphasis  on  automated  solutions  to  an  increasing 
level  of  daily  business  activities,  requires  that  the  man-machine  interface  become  even 
more  powerful,  intuitive  and  easier  to  use  than  previously  required. 
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II.  Currently  Available  Man-Machine  Interface  Technology. 


This  section  will  address  the  capabilities  and  limitations  of  a  number  of  man-machine 
interface  technologies  that  are  currently  available. 

A.  Capabilities  of  current  man-machine  interface  technology. 

Hardware  or  mechanical  interfaces  include  a  plethora  of  technologies  from 
keyboards,  number  pads  and  monitors  on  the  low  end  of  the  technology,  to  the  more 
technically  sophisticated  options  such  as  optical  character  readers/  scanners,  voice 
recognition/synthesis,  and  electrical/  mechanical  pointing  devices  such  as  the  mouse, 
light  pens,  touch  sensitive  screens,  writing  tablets,  trackballs  and  joysticks.  The 
development  of  these  devices  illustrates  a  recognized  need  for  making  the 
human-machine  interaction  both  physically  and  psychologically  less  arduous.  Most  of 
these  require  software  interface  counterparts  to  augment  the  ease  of  use.  An  example  is 
the  use  of  menu  screens  or  graphic  icons  in  conjunction  with  pointing  devices  such  as 
the  mouse,  light  pen  or  touch  sensitive  screen.  Voice  recognition  and  synthesis  interfaces 
have  only  recently  become  commercially  available.  The  former  is  a  interface  method  in 
which  human  speech  is  digitized  and  translated  into  machine  readable  instructions. 
Examples  of  this  technology  include  Kurzweil’s  Voicesystem  and  Voiceterminal  products 
and  Roar  Technology’s  Voicekey  board,  each  of  which  support  a  vocabulary  of  over  1000 
words.  Speech  syntheses  is  an  output  interface  in  which  the  machine  communicates  with 
the  user  by  translating  machine  readable  information  into  understandable  human  speech. 

Software/graphics  interfaces  include  a  variety  of  approaches  to  creating  a  more 
’transparent’  environment  for  the  user —  avoiding  (as  much  as  possible)  the  need  for 
interacting  with  the  housekeeping  functions  of  the  operating  system/hardware  through  the 
use  of  standardized,  reasonably  intuitive  graphics  through  which  the  user  interacts  with 
the  application.  The  development  of  windowing  interfaces — a  bit  mapped  display  of 
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variable  size,  shape  and  location  on  the  screen,  in  which  information  is  displayed — 
reflects  the  increasing  significance  placed  on  presenting  an  intuitive,  easy  to  use  and 
understandable  interface.  MS-Windows  and  Presentation  Manager  for  personal 
computers  and  various  X-Windows  based  graphics  for  Unix  machines  are  examples  of 
this  technology.  The  integration  of  these  software  interfaces  with  the  pointing  devices 
described  above  further  enhances  the  growing  ease  of  use  of  computers  for  end  users. 
Additionally,  an  increasing  number  of  products  are  beginning  to  introduce  limited 
’’natural  language”  front  ends  either  to  the  application  or  the  interface  environment  (eg 
windows)  to  reduce  the  user’s  requirement  for  detailed  knowledge  of  the  syntax  required 
to  use  the  operating  system  or  to  execute  the  application. 

B.  Limitations/quality  of  current  man-machine  interfaces. 

Optical/Mechanical  Interfaces  Input  devices.  Despite  the  existence  of  mechanical 
interface  devices  for  10-15  years,  widespread  support  of  these  products  by  software 
developers  has  limited  both  the  acceptance  and  innovative  use  of  these.  An  example  of 
the  use  of  the  mouse  as  a  pointing  device  clearly  illustrates  this  point.  The  mouse  is  a 
de  facto  standard  on  most  high  end  workstations,  generally  integrated  both  within  the 
system  interface  and  the  application  environment.  Within  the  general  purpose  personal 
computer  environment,  however,  only  a  very  limited  number  of  applications  (Macintosh, 
GEM)  have  been  developed  with  an  eye  toward  the  use  of  these  devices.  The  Macintosh 
line  of  computers  (and  support  software)  has  made  significant  inroads  in  developing 
credibility  for  this  technology.  Optical  input  devices  provide  a  useful  data  transfer 
alternative  to  keyboard  input  from  a  paper  medium  to  electronic  form.  The  fastest  and 
most  economical  way  to  perform  this  transfer  is  to  convert  non-electronic  data  to  a 
digital  format  through  scanning.  The  three  primary  methods  of  optical  recognition 
include  bar  coding,  scanning  and  optical  character  reading.  Bar  coding  is  a  widely  used 
method  that  employs  a  pattern  of  dark  and  light  lines  of  different  widths  as  code  for 


information.  It  is  particularly  popular  for  inventory  functions.  The  remaining  two 
methods  are  digital  and  digitized  scanning.  Digital  scanners  are  called  optical  character 
readers  (OCR).  They  read  individual  characters  and  encode  them  into  binary, 
machine-readable  form.  OCR’s  distinguish  between  different  characters  and  fonts  and 
can  generally  adapt  the  tran sfered  data  to  popular  word  processing  packages.  Digitized 
scanners  convert  a  document  into  picture  elements  called  pixels.  This  data  must  be 
indexed  for  retrieval  because  it  does  not  distinguish  between  individual  characters. 
Scanning  is  the  faster,  but  less  accurate  method,  and  OCR  is  in  effect  an 
implementation  of  the  scanner  technology,  in  which  sophisticated  algorithms  are 
employed  to  determine  the  specific  character  being  translated.  Optical  scanning  is  not  a 
mature  technology.  A  number  of  limitations  affect  the  widespread  use  of  this  technology, 
including: 

a.  a  lack  of  standards  for  image  scanning  output  formats  or  the  scanned  image 
storage  format, 

b.  scanning  devices  generally  run  software  that  serve  a  limited  number  of 
commercial  applications, 

c.  digital  scanners  are  limited  to  the  recognition  of  specific  fonts  and  styles 
(proportional  and  dot  matrix  print  are  virtually  unreadable)  and  tend  to  be  error  prone, 

d.  the  storage  requirements  for  digitized  documents  is  significant. 

Speech  recognition  systems  have  made  significant  progress  in  just  the  past  2-3 
years.  Impressive  strides  have  been  made,  particularly  in  increasing  the  size  of  usable 
vocabularies  for  isolated  (carefully  articulated  single  words)  speech.  The  importance  and 
progress  of  this  technology  are  well  illustrated  in  the  recently  introduced  NEXT 
computer  which  employs  voice  I/O  technologies  for  voice  mail.  Despite  this,  significant 
improvements  are  required  both  in  expanding  the  vocabulary  of  the  system  (currently 
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limited  to  approximately  1000  words)  and  more  accurate  recognition  of  continuous  or 
connected  speech  input  both  from  different  users  and  in  noisy  environments.  Output 
Devices.  Despite  the  growing  popularity  of  Enhanced  Graphics  Adapter  (EGA)  and 
Video  Graphics  Array  (VGA)  color  medium  to  high-resolution  graphics  monitors,  the 
most  common  output  interface  device  on  general  purpose  personal  computers  remains 
(primarily  due  to  low  price)  the  low-to-medium  resolution  (640  x  400  or  less  pixel 
resolution)  monochrome  monitor.  Use  of  the  color  graphics  adapter  (CGA)  monitor  is 
also  relatively  widespread. 

The  predominant  low  end  system  interface  continues  to  be  either  arcane  operating 
system  command  languages  (DOS,  UNIX)  or  menu-driven  interfaces.  The  former 
requires  a  significant  level  of  technical  knowledge  to  execute  applications  and  perform 
housekeeping  activities.  The  latter  severely  limits  the  user’s  ability  to  concisely 
communicate  requirements  to  the  system  that  are  not  exactly  delineated  by  the  menu 
interface.  The  current  generation  of  high  end  system  interfaces  are  best  characterized  by 
Apple  Computers  Macintosh  line,  the  Digital  GEM  and  Microsoft  Windows 
environments.  The  integrated  use  of  somewhat  descriptive  icons  with  pointing  devices 
and  a  windowing  environment  has  set  a  direction  and  minimum  standard  for  future 
industry  developments.  The  limited  widespread  exploitation  of  these  features  by 
application  developers  is  a  significant  limitation  to  acceptance  and  use.  Additionally,  the 
current  generation  of  windowing  interfaces  presently  lack  the  ability  to  tie  together 
applications. 

C.  Application  to  the  IMA. 

The  increased  use  of  currently  available  interface  technology  can  effect  significant 
productivity  improvement  by  making  automation  resources  easier  to  learn  and  use. 
Intuitively,  this  would  lead  to  reduced  time  and  costs  for  training.  Continued 
improvements  in  the  design  of  interfaces  is  increasing  this  level  of  transparency  of  the 
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underlying  operating  systems,  hardware  and  programing  languages.  Commercial  (if 
non-standard)  program  application  interfaces  are  currently  available  to  tie  disparate 
applications  together  under  a  single  global  interface.  Examples  of  this  include  the 
integrated  business  software  from  Lotus  (Symphony),  Ashton-Tate  (Framework)  and 
MDBS  (Knowledgeman  and  Guru).  The  current  generation  of  optical  scanner  devices 
can  provide  significant  savings  and  increased  productivity  in  data  translation  efforts. 

III.  Man-Machine  Interface  Technologies  in  the  Near  Term  (1995) 

This  section  addresses  new  developments  in  man-machine  interfaces  that  will  make 
a  significant  impact  within  the  next  3-8  years.  Realistically,  much  of  the  MMI 
technology  that  is  beginning  to  be  introduced  or  become  commercially  available  today 
will  not  be  in  widespread  use,  particularly  in  the  Army,  for  another  2-5  years.  This  lag 
time  in  the  transfer  of  technology  from  the  business  community  into  the  Army  is  due 
primarily  to  the  procurement  processes  involved  in  making  this  transfer. 

A.  Capabilities  of  man-machine  technologies  in  the  near  term. 

1.  Hardware  interfaces. 

The  most  significant  improvement  in  interface  technology  will  be  the  commonplace 
existence  of  integrated  standard  tools  at  each  workstation,  much  like  those  found  on 
powerful  workstations  in  current  research  and  academic  facilities.  Many  of  the  tools 
prevalent  on  workstations  today  will  become  commonplace  standards  for  the 
office/administrative  workstation.  These  include  a  mechanical  pointing  device  such  as 
the  trackball  or  mouse,  a  robust  voice  recognition  system  (limited  to  a  small  command 
set)  for  vocal  input  directly  into  the  application  (or  system  interface),  high  resolution 
(1024  x  1024  or  greater)  full  page,  color  display  screens,  local  area  network  (LAN) 
access  to  both  document  scanning  and  optical  character  reader  equipment  (input)  and 
inexpensive,  high  speed  laser-quality  printers  or  color  screen  projection  equipment 


-  7  - 


(output).  Each  of  these  technology  areas  are  detailed  below. 

a.  Optical  I/O  Technology.  Optical  I/O  technologies  fall  into  two  broad  categories: 
1)  Direct  human-machine  interfaces  in  which  optical  transceivers  are  used  to  detect  and 
translate  user  eye  movement  to  specific  screen  locations  (coordinated  with  either  vocally 
or  mechanically  directed  execution)  and  2)  optical  scanning  devices  which  translate 
information  from  one  media  (usually  paper)  into  an  electronic  form  fur  manipulation. 
Prototype  direct  optical  interfaces  are  currently  under  development.  Analytics  has  been 
working  on  an  Ocular  Attention-Sensing  Interface  System  (OASIS)  for  over  3  1/2  years. 
OASIS  has  been  used  in  conjunction  with  voice-recognition  technology  in  a  variety  of 
functions  including  vehicle  and  robot  control,  and  inputting  and  manipulation  of 
simulations  and  conventional  business  software  applications  (databases  and 
spreadsheets).  These  devices  will  be  commercially  available  in  the  next  3-4  years, 
however,  the  hardware  requirements  for  the  transceivers  and  calibration  equipment  will 
limit  their  use,  at  least  in  the  near  term,  to  high-end  workstations.  Examples  of  optical 
scanning  devices  include  the  document  scanners  and  optical  character  readers  (OCR) 
previously  discussed.  The  scanner  will  become  a  generic  input  device  that  can  be  used 
with  any  software  following  the  consolidation  of  image  scanning  and  OCR  technologies 
with  facsimile  transmission  technology. 

b.  Voice  I/O  Technology.  Both  voice  recognition  and  voice  synthesis  systems  will 
be  well  established  commercial  products.  Voice  recognition  technology  will  be  well 
established  in  manufacturing  and  process  control  environments.  Relatively  robust 
isolated  and  connected  speech  interfaces  will  be  commercially  available  for  general 
purpose  business  applications  (database,  graphics,  communications)  and  will  be  widely 
integrated  with  digital  voice  communications  (voicemail,  digitized  voice-print  security). 

2.  Software  Interfaces. 

The  software  interfaces  prevalent  on  general  purpose  computer  systems  in  the  near 
term  will  strongly  resemble  the  environments  found  on  the  current  generation  of  top-end, 
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high  powered  workstations  such  as  the  Symbolics,  Apollo  and  larger  Vax  and  Sun 
systems.  PC  systems  (predominately  Motorola  68000+  and  Intel  80386  microprocessor- 
based  equipment)  will  employ  Microsoft  Windows/Presentation  Manager  style  windowing 
environments  that  employ  descriptive  icon  graphics  to  assist  the  user  in  navigating  the 
system.  Some  of  these  icons  will  have  the  appearance  of  familiar  equipment  and  employ 
graphics  that  resemble  common  objects  and  tools  used  in  day  to  day  activity.  The  window 
interface  will  possess  a  greater  ability  to  tie  applications  together.  The  operating  system 
command  language  will  become  both  less  arcane  and  less  available  to  the  end  user  due 
to  the  insulating  effect  of  natural  language  and  graphic  interfaces  that  will  make  the 
operating  system  more  transparent  to  the  user.  Underlying  these  graphic  interfaces  will 
be  a  growing  level  of  computer-as-assistant  role  that  will  integrate  computer  assistant 
instruction  modules  controlled  by  expert/knowledge  based  modules  with  the  system 
interface.  This  assistant  role  of  the  interface  environment  will  provide  the  user  with 
assistance  in  selection  of  correct  tools  for  a  particular  application,  instruction  for 
employing  the  tool  and  guidance  during  the  use  of  the  tool  (e.g.  on-line  syntax/semantics 
checking,  hueristic-based  answering  capibility  in  response  to  user-generated  “how"  and 
"why”  questions). 

High  density  graphic  screens  will  provide  the  necessary  palette  for  animated 
presentation  graphics,  sophisticated  development  tools  (computer  assisted  design  (CAD), 
computer  assisted  software  engineering  (CASE),  decision  support  systems  (DSS))  and 
visually  ’sharing’  activity  between  activities  running  in  separate  windows.  These  will  be 
enhanced  for  audience  viewing  (eg.  formal  presentations  or  training)  with 
high-resolution  screen  projection  systems. 

B.  Performance/Quality  of  man-machine  interfaces  in  the  near  term. 

The  greatest  potential  for  advanced  man-machine  interfaces  will  result  from  the 
integration  of  multiple  interface  technologies  that  overcome  each  other’s  limitations. 
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The  user  is  provided  with  a  kind  of  three-dimensional  simulation  of  the  familiar  office 
environment  through  the  integration  of  these  technologies  enhanced  with  techniques  from 
artificial  intelligence  (e.g.  expert  system  control  of  the  various  elements  of  the  interface), 
object  oriented  programming  (optimizing  re-use  of  components,  providing  an  integrating 
environment  for  diverse  components)  and  graphics  methodologies. 

C.  Application  to  the  IMA. 

The  widespread  use  of  ’point  and  execute’  devices,  robust  voice  recognition  and 
synthesis  tailored  to  thespecific  application,  standardized  window-based,  iconic  graphics 
in  a  multi-processor  environment  will  showcase  the  continued  transition  of  the  interface 
technology  currently  available  on  hi-powered  workstations  to  the  end  user/  developer 
environment.  These  capabilities  will  provide  increasing  consistency  in  user-machine 
dialogue,  minimizing  the  requirement  to  retrain  users  between  different  systems  and 
reducing  the  cost  of  training  new  users.  Reduced  costs  in  data  translation  between 
disparate  media  (e.g.  paper  and  electronic)  will  continue  to  be  realized  as  scanner/OCR 
technologies  continue  to  mature. 

IV.  Man-Machine  Interface  Technologies  in  the  Long  Term  (2010) 

This  section  will  examine  developing  technology  that  will  have  an  impact  on 
man-machine  interface  systems  in  the  early  21st  century. 

A.  Capabilities  of  man-machine  technologies  in  the  long  term. 

1.  Hardware 

In  the  long  term,  an  increased  sophistication  of  OCR/Text  recognition  will  become 
available  through  the  use  of  hueristic/  cognitive  based  algorithms.  High  resolution 
displays  will  be  common-place  and  will  integrate  automation  with  communications, 
particularly  digitized  television  images  within  windows  on  the  display.  Continuous 
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speech  voice  recognition  interfaces  will  virtually  replace  keyboards  as  the  primary 
human-computer  interface.  Current  research  in  electroencephalographic  control  of 
interface  devices  —  mechanisms  that  are  activated  by  devices  that  detect  electrical  brain 
activity  and  translate  this  activity  into  simple  commands —  will  produce  the  initial 
generation  of  commercial  electroencephalographic  interface  products.  These  will 
probably  become  available  first  in  the  area  of  vehicle  and  robot  control  and  will  begin  to 
be  explored  for  at  least  simulation  programs  and  control  of  icon  graphics. 

2.  Software. 

The  greatest  advances  will  be  in  the  use  of  increasingly  sophisticated  natural 
language  capabilities  integrated  with  both  the  system  and  application  interfaces.  Direct 
access  to  the  operating  system  and  control  of  devices  through  command  languages  will 
become  virtually  obsolete.  The  increasing  distributed  nature  of  automation  will  require 
further  sophistication  in  system-level  interfaces  that  insulate  the  user  from  a  requirement 
to  know  any  details  about  the  distributed  nature  of  the  environment.  Access  to  remote 
databases,  equipment,  systems  etc.,  will  be  a  function  of  virtual  windows  driven  by 
underlying  knowledge -based  software  rather  than  user  control.  As  the  system 
environment  continues  to  become  more  complex,  the  interface  mechanisms  will  present 
an  increasingly  more  intuitive,  non-technical  user  environment.  The  primary 
screen/system  interface  will  be  an  integrating  environment  to  provide  applications  access, 
training,  presentation  graphics  and  activity  assistance  support  to  the  user. 

B.  Application  to  the  IMA. 

Software  interfaces  will  provide  a  de  facto  standard  to  tie  applications  together  and 
provide  a  common  environment  for  a  broad  base  of  users.  This  common  environment, 
supported  by  both  knowledge  based  control  and  explanation  features  and  robust  natural 
language  capabilities  will  further  reduce  the  requirement  for  retraining  users  from 
system  to  system  (and  associated  costs),  minimizes  the  impact  of  employing  disparate 
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applications  packages  from  different  vendors,  and  provides  a  common  and  familiar 
interface  amoung  different  applications.  This  environment  will  provide  the  mechanism 
to  integrate  the  entire  spectrum  of  the  IMA,  including  printing  and  publishing,  records 
management  and  audio-visual,  communications  (digitized  voice  mail,  electronic  mail, 
network  interaction)  and  automation  requirements.  The  widespread  use  of  continuous 
voice  technology  especially  coupled  with  ocular  interface  devices,  will  eliminate  the 
requirements  for  cumbersome  keystrokes  or  button  oriented  command  and  control 
systems  to  operate  an  application  or  operating  system.  This  will  in  turn,  further  impact 
on  both  reducing  training  and  operational  costs  and  enhancing  user  productivity. 
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